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Abstract — A universal algorithm for sequential data compres- 
sion is presented. Its performance is investigated with respect to 
a n on probabilistic model of constrained sources. The compression 
ratio achieved by the proposed universal code uniformly ap- 
proaches the lower bounds on the compression ratios attainable by 
block-to-variable codes and variable- to- block codes designed to 
match a completely specified source. 

I. INTRODUCTION 

IN MANY situations arising in digital com- 
munications and data processing, the encountered 
strings of data display various structural regularities or are 
otherwise subject to certain constraints, thereby allowing 
for storage and time-saving techniques of data compres- 
sion. Given a discrete data source, the problem of data 
compression is first to identify the limitations of the source, 
and second to devise a coding scheme which, subject to 
certain performance criteria, will best compress the given 
source. 

Once the relevant source parameters have been identi- 
fied, the problem reduces to one of minimum-redundancy 
coding. This phase of the problem has received extensive 
treatment in the literature P]-[7]. 

When no a priori knowledge of the source characteristics 
is available, and if statistical tests are either impossible or 
unreliable, the problem of data compression becomes 
considerably more complicated. In order to overcome these 
difficulties one must resort to universal coding schemes 
whereby the coding process is interlaced with a learning 
process for the varying source characteristics [8], [9]. Such 
coding schemes inevitably require a larger working mem- 
ory space and generally employ performance criteria that 
are appropriate for a wide variety of sources. 

In this paper, we describe a universal coding scheme 
which can be applied to any discrete source and whose 
performance is comparable to certain optimal fixed code 
book schemes designed for completely specified sources. 
For lack of adequate criteria, we do not attempt to rank the 
proposed scheme with respect to other possible universal 
coding schemes. Instead, for the broad class of sources 
defined in Section III, we derive upper bounds on the 
compression efficiency attainable with full a priori 
knowledge of the source by fixed code book schemes, and 
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then show that the efficiency of our universal code with no 
a priori knowledge of the source approaches those 
bounds. 

The proposed compression algorithm is an adaptation 
of a simple copying procedure discussed recently [10] in 
a study on the complexity of finite sequences. Basically, 
we employ the concept of encoding future segments of the 
source-output via maximum-length copying from a buffer 
containing the recent past output. The transmitted 
codeword consists of the buffer address and the length of 
the copied segment. With a predetermined initial load of 
the buffer and the information contained in the codewords, 
the source data can readily be reconstructed at the de- 
coding end of the process. 

The main drawback of the proposed algorithm is its 
susceptibility to error propagation in the event of a channel 
error. 

II. The Compression Algorithm 

The proposed compression algorithm consists of a rule 
for parsing strings of symbols from a finite alphabet A into 
substrings, or words, whose lengths do not exceed a pre- 
scribed integer L s , and a coding scheme which maps these 
substrings sequentially into uniquely. decipherable code- 
words of fixed length L c over the same alphabet A. 

The word-length bounds L s and L c allow for bounded- 
delay encoding and decoding, and they are related by 



L e « 1 + Hog (n - LJ1 + flog L s l 



(1) 



where [x~\ is the least integer not smaller than x, the log- 
arithm base is the cardinality a of the alphabet A , and n 
is the length of a buffer, employed at the encoding end of 
the process, which stores the latest n symbols emitted by 
the source. The exact relationship between n and L s is 
discussed in Section EL Typically j n ^ L s a hLt , where 0 
< h < 1. For on-line decoding, a buffer of similar length has 
to be employed at the decoding end also. 

To describe the exact mechanics of the parsing and 
coding procedures, we need some preparation by way of 
notation and definitions. 

Consider a finite alphabet A of a symbols, say A = 
{0,1, - • • ,a — 1). A string, or word, S of length £{S) = k over 
A is an ordered k -tuple S = s\S2 * • • Sk of symbols from A. 
To indicate a substring of S which starts at position i and 
ends at position y, we write S(iJ). When i < j, S(iJ) = 
SiSg+i • • • Sj, but when * > j, we take S(iJ) = A, the null 
string of length zero. 

The concatenation of strings Q and R forms a new string 
S = QR; if HQ) = k and £{R) = m, then £(S) = k + m, Q 
= S(l,fc), and R = S(k + 1, k + m). For each;, 0 < < 
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t(S) f 5(1 j) is called a prefix of 5; 5(1 j) is a proper prefix 
of 5 if j<t(S). 

Given a proper prefix 5(1 J) of a string 5 and a positive 
integer i such that i < j t let L(i) denote the largest non- 
negative integer t < t(S) - j such that 

and let p be a position of 5(1 J) for which 
L{p) « max \Ui)\. 

l<i<y 

The substring 50" + lj + £(P» ° f 5 is called the repro- 
ducible extension of 5(lj) into 5, : and the integer p is 
called the pointer of t\\e reproduction. For example, if 5 
= 00101011 and ; = 3, then L(l) = 1 since 50* + 1 J + D = 
5(1,1) but 50' + lj + 2) ^ 5(1,2). Similarly, L (2) = 4 and 
L(3) = 0. Hence, 5(3 + 1,3 + 4) = 0101 is the reproducible 
extension of 5(1,3) = 001 into 5 with pointer p = 2. 

Now, to describe the encoding process, let S = Si$ 2 • • • 
denote the string of symbols emitted by the source. The 
sequential encoding of 5 entails parsing 5 into successive 
source words, 5 = 5i5 2 • • • , and assigning a codeword C t 
for each Si. For bounded-delay encoding, the length t t of 
each 5;. is at most equal to a predetermined parameter L s , 
while each C; is of fixed length L c as given by (1). 

To initiate the encoding process, we assume that the 
output 5 of the source was preceded by a string Z of n - 
L s zeros, and we store the string B x = ZS{\JL 8 ) in ti\e 
buffer. If 5(1 j) is the reproducible extension of Z into 
Z5(1,L, - 1), then S x = S{lJ ,+ D and i x = j + 1. To de- 
termine the next source word, we shift out the first t x 
symbols from the buffer and feed into it the next t x sym- 
bols of 5 to obtain the string B 2 - B x {t x + l,n)5(L, + 1, 
L s + £i). Now we look for the reproducible extension E of 
B 2 (l,n - L s ) into B 2 (l,n - 1), and set 5 2 = Es, where s is 
the symbol next tp E jn B 2 . In general, if Bi denotes the 
string of n source symbols stored in the buffer when we are 
ready to determine the tth source word 5,-, the successive 
encoding steps can be formally described as follows. 

1) Initially, set B x - 0 n -^5(l,LJ, i.e., the all-zero 
string of length n - L B followed by the first L s symbols of 
5. 

2) Having determined Bi, i > 1, set 

. Si = Bi(n - L 8 + l,n - L, + ii). 

where the prefix of length ti - 1 of 5; is the reproducible 
extension of fy(l,n - L s ) into B;(l,a - 1). 

3) If pi is the reproduction pointer used to determine 
5 t , then the codeword C t for 5, is given by 

where Cn is the radix-a representation of Pi - 1 with 
£{C n ) = flog {n - L,)l, C l2 is the radix-a representation 
of ti - 1 with * (C l2 ) = Tlog L s 1 , and C l3 is the last symbol 
of 5 : , i.e., the symbol occupying position n - L* +' of B t -. 
The total length of C 4 - is given by 

Hd) = Rog (n - L,)l + Hog L s l + 1 

in accordance with (1). 
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4) To update the contents of the buffer, shift out the 
symbols occupying the first ti positions of the buffer while 
feeding in the next ti symbols from the source to obtain 

B I+1 = BAti + l,n)S(hi + 1A- + ti). 

where hi is the position of 5 occupied by the last symbol 
ofB,-. 

. This completes the description of the encoding process. 
It is easy to verify that the parsing rule defined by (2) 
guarantees a bounded, positive source word length in each 
iteration; in fact, 1 < ti < L s for each i thus allowing for 
a radix-a representation of t- t — 1 with flog L s ] symbols 
from A. Also, since 1 < p, < n - L* for each i, it is possible 
to represent pi — 1 with flog (n — L s )l symbols from A. 

Decoding can be performed simply by reversing the 
encoding process. Here we employ a buffer of length n — 
L.„. to store the latest decoded source symbols. Initially, the 
buffer is loaded with n - L, zeros. If D; = d x d 2 - * • d n -L„ 
denotes the contents of the buffer after C t _i has been de- 
coded into 5/-i, then 

S,-_i = Di(n - L s - *£-i + l,n - L,), 

where — t(Si- x ), and where Z),+i can be obtained 
from Di and C t - as follows. 

Determine p; - 1 and ti - I from the first flog (n - L„)l 
and the next flog L s l symbols of d. Then, apply ti — 1 
shifts while feeding the contents of stage p; into stage n - 
L s . The first of these shifts will change the buffer contents 
from Di to 

D\" = d 2 d 3 • • • d n -udp t * d[*W • * * <ft'i/.. 

Similarly, if ; < ti — 1, the ;th shift will transform oV" l> 
- d^* l) dV'- l) ...d ( /rI> into D\ J) = d^'- 1 ^ 1 '--- 
dfcfidK"" = A f ter these ^ ; - 1 

shifts are completed, shift once more, while feeding the last 
symbol of d into stage n - L 8 of the buffer. It is easy to 
verify that the resulting load of the buffer contains 5/ in 
its last ti = t(Si) positions. 

Trie following example will serve to illustrate the me- 
chanics of the algorithm. Consider the ternary (a = 3) 
input string 

5 = 001010210210212021021200 

and an encoder with parameters L s = 9 and n = 18. (These 
parameters were chosen to simplify the illustration; they 
do not reflect the design considerations to be discussed in 
Section III.) According to (1), the corresponding codeword 
length is given by 

L c = 1 + log 3 (18 - 9) + Log 3 9 = 5. 

Initially, the buffer is loaded with n - L s = 9 zeros, fol- 
lowed by the first L s = 9 digits of 5, namely, 

Bi = 0 0 0 0 0 000 0 001010210. 

n - L s = 9 L 3 = 9 

To determine the first source word Si, we have to find the 
longest prefix ZM1Q.9 + ti - D of 

Bi(10,17) = 00101021 
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which matches a substring of Bi that starts in position p i 
< 9 and then set S x = Bi(10,9 + £1). It is easily seen that 
the longest match in this case is Bi(10,ll) = 00, and hence 
Si = 001 and l\ = 3. The pointer pi for this step can be any 
integer between one and nine; we choose to set pi = 9. The 
two-°digit radix-3 representation of Pi - 1 is Cu = 22, and 
that of 1 1 - 1 is C l2 = 02. Since C i3 is always equal to the 
last symbol of S £f the codeword for Si is given by Ci = 
22021. 

To obtain the buffer load B 2 for the second step, we shift 
out the first ti = Z digits of B x and feed in the next 3 digits 
S (10,12) = 210 of the input string S. The details of steps 
2, 3, and 4 are tabulated below, where pointer positions are 
indicated by arrows and where the source words Si are 
indicated by the italic substring of the corresponding 
buffer load Bi 

i 

B 2 = 000000001 01 0210210, 
1 

B z = 000010102102102/2 0, 
I 
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Typically, such a source <r is defined by specifying a fi- 
nite set of strings over A which are forbidden to appear as* 
substrings of elements belonging to <r, and therefore v(m) 
< a m for all m exceeding some m 0 - 

With every source <r, we associate a sequence 
h(2),... of parameters, called the h- parameters of <t, 
where 1 



C 2 = 21102 
C 3 « 20212 



B 4 = 21021021202102120 0, C 4 = 02220. 

in. Compression of Constrained Sources 

In this section, we investigate the performance of the 
proposed compression algorithm with respect to a non- 
probabilistic model of constrained information sources. 
After defining the source model, we derive lower bounds 
on the compression ratios attainable by block-to-variable 
and variable-to-block codes under full knowledge of the 
source, and then show that the compression ratio achieved 
by our universal code approaches these bounds. 

A. Definition of the Source Model 

Let A = (0,1, • • • ,a - 1| be the given a-symbol alphabet, 
and let A * denote the set of all finite strings over A. Given 
a string 5E^* and a positive integer m < £(S) y let S\m\ 
denote the set of all substrings of length m contained in S, 
and let S(m) denote the cardinality of S(m}. That is, 



Sim) 



f(S)-m 

U S(i + U' + m) 

i-0 



and 



S(m) = \S[m\\. 



h{m) = — log a(m), 
m 



m = 1,2, 



(2) 



It is clear that 0 < h(m) < 1 for all m and, by 2) it is also 
clear that mh(m) is a nondecreasing function of m. The . 
sequence of /i-parameters, however, is usually nonin-^, 
creasing in m. To avoid any possible confusion in the se- 
quel, we postulate this property as an additional defining 
property of a source. Namely, we require 

4) h(m) = l/m log <r(m) is a nonincreasing function of 
m. 

B. Some Lower Bounds on the Compression Ratio 

Consider a compression coding scheme for a source a 
which employs a block- to- variable {BV) code book of M 
pairs (Xi,Yi) of words over A, with €(Xi) = L f or i - 
1,2, ■ • • ,M. The encoding of an infinitely long string S £ * 
o-'by such a code is carried out by first parsing S into blocks 
of length L, and then replacing each block X; by the cor- 
responding codeword Yi. It is assumed, of course, that the 
code book is exhaustive with respect to <r and uniquely 
decipherable [2j. Hence, we must have 

\Xi\ 



and 



M = <x(L) = a Lh{L \ 



max \t(Yi)\> logAf = LML). 

l<i<M 



(3) 



(4) 



Pi =' 



The compression ratio pi associated with the ith word- 
pair of the code is given by 

*(Y«) 
L ' 

The BV compression ratio, pbv(vM), of the source c. 
is defined as the minimax value of pu where the maximi- 
zation is over all word-pairs of a given code, and the min- 
imization is over the set C B v{*M) of all BV code books 
consisting of M word-pairs. Thus, 



Given a subset <r of A*, let 

a\m\ = \S e <r\t{S) = ml 

and let a(m) denote the cardinality of <r{m\. 

A subset <r of A * is called a source if the following three 
properties hold: 

1) A C <r (i.e., <r contains all the unit length strings), 

2) S £ ° implies SS £ <r, 

3) S £ (J implies S\m] C <r[m). 



p BV {a,M) = niin max 



L 



> logM = LHL) a h ( L) 
~ L L 



1 Throughout this paper, log x means the base-or logarithm of x. 
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For later reference, we record this result in the following 
lemma. 
Lemma 1: 



where 



Lh(L) = logM. 

Now, consider a compression scheme which employs a 
variable-to-block (VB) code book of M word-pairs {X it 
with t( Yi) = L f or all i - 1,2, - • • ,Af . In this case, the 
compression ratio associated with the tth word-pair is 
given by 



Pi 



and similarly, the VB compression ratio pvb(^M) of a is 
defined as the minimax value of p; over all word-pairs and 
over the set Cvb(gM) of VB code books with M word- 
pairs. 

Lemma 2: 



p VR {o,M)±h{L M ) 



where 



L M = max \e\M > a(£)\. 

Proof: We may assume, without loss of generality, that 
in every code under consideration 

£{X X ) <HX 2 ) <.--<£(X M ), 

and hence for each C £ CvaivM)* 

L(C) 



max pi(C) 



(5) 



e{xj 

Since C is exhaustive with respect to <r, we have 

M > <r(*i), (6) 

where *i = ^(Xi); and since C is uniquely decipherable, 
we have 

L(C)>logM. (7) 

From the definition of Lm. inequality (6), and the nonde- 
cr easing property of <r(t), we obtain 

e x <L M . (8) 

From (5), (7), and (8), we have 

max pi(C) > — ; 

Lm 

and since 

M > <j(L m ) = ot L ^ LM) , 
it follows that for each C £ Cvb&M), 
max pi(C) >h(L M ). 

Q.E.D. 
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1) Since the value of L in the context of Lemma 1 
satisfies the definition of Lm as given in Lemma 2, it fol- 
lows that the bounds of both lemmas are essentially the 
same, despite the basic difference between the respective 
coding schemes. 

2) By 2), the second defining property of a source, the 
per-word bounds derived above apply to indefinitely long 
messages as well, since the whole message may consist of 
repeated appearances of the same worst case word. 

3) By 4), the nonincreasing property of the /i-pa- 
rameters, the form of the derived bounds confirms the 
intuitive expectation that an increase in the size M of the 
employed code book causes a decrease in the lower bound 
on the attainable compression ratio. 

C. Performance of the Proposed Algorithm 

We proceed now to derive an upper bound on the com- 
pression ratio attainable by the algorithm of Section II. To 
this end, we consider the. worst case source message of 
length n — L sy where n is the prospective buffer length and 
L s is the maximal word-length. The bound obtained for 
this case will obviously apply to ail messages of length n 
— L s or greater. 

First, we assume that only the h -parameters of the 
source under consideration are known to the designer of 
the encoder. Later, when we discuss the universal perfor- 
mance of the proposed algorithm, we will show that even 
this restricted a priori knowledge of the source is actually 
unessential. 

We begin by choosing the buffer length n to be an inte- 
ger of the form 

n a £ mct m + £ m<j{t) + 0? + l)(ivYn + 1), 

m- 1 m= X+l 



(9) 



where 



Nt+i = £ (i - m)a" + £ (t - «)*(/), (10) 

m~ 1 m=X+l 

A = U/i(/)J, the integer part of log a(t) = eh(£), and 

e = L 8 -1. (11) 

The specific value of the parameter L s is left for later 
determination (see the first remark following the proof 
of Theorem 1). The reasoning that motivates the given 
form of n will become clear from subsequent deriva- 
tions. 

Consider a string S E o\n — L s ), and let N(S) denote the 
number of words into which S is parsed by the algorithm 
during the encoding process. Recalling that each of these 
words is mapped into a codeword of fixed length L c (see 
(1)), it follows that the compression ratio p{S) associated 
with the string S is given by 



P(S) = 



n — L s 
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Hence, the compression ratio p attainable by the algorithm or 

for a given source a is N < N' = n ~ Ls = n ~~ L ?- . (16) 

t £ Lt s — 1 

Lc -N, (12) 



9 n-L $ ' Hence, from (12) and (16), we have 

where P < — = ^ c . (17) 

£ Lg 1 

N = se?\^U\ Note that despite the rater rudimentary overestimation 

r \ u u ,w M(Q\-N and suDDOse of N by IV', the upper bound of (17) is quite tight, since the 

<t„for«)m C .<;<N.lh«i9i < Q, - (N °" „ "T* T W«m 1. If the buffer length » for « source with 

< L a , the longest substring in the buffer that precedes Q y - p < /i (L s - 1 ) + € {L s ), 
and'is a prefix of Qj must be of length £ (Q } ) - 1.) 

Denoting by K„ the number of Q., 1 < iS AT - 1. of where 

length m, 1 < m < L„ we have = /g + 3 log (jL , _ i) + i og ^Y 

N = 1 + m ' Proo/: From (1) we have 

By the above argument, and by property 3) of the source, Lc = i + ri og l s ^ + Hog (n - L s )1 

we have < 3 + log (L 8 - 1) + log (» - L,). 

JC m < ,<m>. for 1 < m < / - U - 1- ^ (g) ^ (1Q) we obtain 

Since r x / , x ,^ 

n - L. = / L (/ - "»)« m + E (' _ 

n - L s = /(Q/v) + Z ^m, L— 1 m=x+1 

and » and U are both fixed, it is clear that by overesti- + « m + J w J. 
matine the values of K m for 1 < m. < £ at the expense of 

K, we can only overestimate the value of N. Therefore, and since a™ < *(/), for 1 < m < X, we have 

sinceV(.) S «n + 1) and = * we ob- ^ _ £ (, - „ + 1> - * + lM/>. 

tain 1 m =i 2 



m — 1 

where 



log (n -L s )<2 log < + log ~1 + ih(t). 



ta m for 1 £ m < X = \.£h(£)i . . 

If » { ^ » (14) Since / = L s - 1, we obtain 

Km U/). forX<m<* 

and L e < 3 + 3 log (L, - 1) + log -f + (L. - l)y»(L, - 1), 



(15) 



V m j L c < (L, - l)[h(L, - 1) + «(L»)]. 
From (14), (15), and (9), we obtain K,+, =N, +1 ,and 

,t0 U M Substituting this result into (17), we obtain the bound of 

N' = N, + 1 + E E -(/) Theorem 1. Q.E.D. 

which, together with (9) and (10), yields ^The va i ue of decreases with L s and, conse- 

x <, ... quently, the compression ratio p approaches the value of 

n-L s -£N'= E " l °' m + E fc( r _ d t he ^-parameter associated with the second 

largest word-length processed by the encoder. Given any 

, v c5 > 0, one can always find the least integer f s such that 

+ Nf + i - t [^E « m + m E +i «r(«j " 0- p _ _ d < fi . The magnitude of the acceptable de- 
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viation 6 determines the operational range of L s , namely, 
L s > i s . 

2) Since our code maps source words of variable 
length at most L 3 into codewords of fixed length L c , we 
adopt as a reference for comparison the best VB code Cy B 
discussed in Subsection III-B. The counterpart of our 
block-length L c is L(C) of (5), and by (7) we can write 



L c « log M ~ L M h(L M ), 



(18) 



where M is the code book size and h(LM) is the lower 
bound (see Lemma 2) on the compression ratio pvb at- 
tainable by Cvs- Since for sufficiently large L st we have 
also 



L c * (L 8 - l)p* (L s - l)h(L a - 1), 



(19) 



it follows that p « pvb- 

We turn now to investigate the universal performance 
of the proposed algorithm, i.e., the performance when no 
a priori knowledge of the source to be compressed is 
available. 

Given 8 X and h x such that 0 < S x < h\ < 1, let £ x be the 
least positive integer satisfying 

$i > e(£ x + 1) = y (3 + 3 log £ x + tee^J^)' 

and let if - a' ,hl and Xi = L^i^iJ- Consider the encoder 
Ei which employs a buffer of length n x = n(h x ,£ x ), ob- 
tained from (9) and (10) by setting £ = £ Xt X = X L , and <r(0 
= and whose maximal word-length L, is equal to £\ + 
1. 

It follows from Theorem 1 that if this encoder is applied 
to a source o x such that h cx (£ x ) = h\ 9 then the resulting 
compression ratio pi(<ri) satisfies 

PifoJ^fe^WO + ai-fci + fc. (20) 

Suppose now that h 1 and 61 were chosen to fit the pre- 
scribed upper value pi — h\ + d x of a prospective com- 
pression ratio range (p 0 ,Pi) with 

0 < 8 X R < po < Pi < 1. * > 1. (21) 
where is an adjustment parameter to be determined 
later. As shown above, the encoder E x is then matched 
exactly to the upper value pi of the prospective compres- 
sion range. In order to accommodate the whole given range, 
we propose to employ a slightly larger encoder Eo whose 
buffer is of length n 0 = n x - £\ + €0, where n x and £ x are 
the parameters of E x , and where £ 0 is an integer, greater 
than ^1, for which the solution ho of the equation 

n x - £\ + £0 = n(hoJo) (22) 

satisfies 



Po - e(*o) <h 0 < po - d£o + 1). 



(23) 



Noting that n 0 - t 0 = n x - £ x is fixed, it is clear from (9) 
and (10) that as t Q increases h 0 decreases; also, (21) and the 
fact that £0 > *\ imply that po - ((£0 + D > Po ~ *(*i + D 
— Po — 5j > 0. Hence, there exist h 0 > 0 and £ 0 > £\ that 
satisfy both (22) and (23). 



In analogy with (20), it is also clear that if Eo is applied 
to a source tr 0 such that h aQ (£o) = h 0t then the resulting 
compression ratio po(^o) satisfies 

Po(*o) ^ Wo) + 5 0 = ho 4- 6 0 , (24) 

where 5 0 = e (^o + 1)« 

From (23) and (24), we have p 0 (<ro) ^ Po - «(^o + D + $o 
= po, and 

60 < po ~ ho < e(^o) ^ etfi + 1) < 6 X < ipo. 

Hence, h 0 can be made arbitrarily close to po, and conse- 
quently, the encoder Eo matches as closely as desired the 
lower end po of the given compression range. 

Theorem 2: Let a be a source for which a matched en- 
coder E achieves a compression ratio p(a) within the range 
(potpi). Then the compression ratio po(<r), achieved for a 
by Eo* satisfies 

po(a) < pM + A, 



where 



4 <M 

^1 



» [rt x 1 

d ~ max t— — - 

\h 0 I -h 0 



(Typically, (h x /h 0 ) > (1/1 - h 0 ) and d = (h x /h 0 ).) 

Proof: To prove the theorem we shall consider the 
obviously worst case of applying Eq to the source o\ whose 
matched encoder E x realizes pi. 

Let po(ffi) denote the compression ratio achievable by 
E 0 when applied to a x . According to (12), we have 

L C Q 



PoM = 



n 0 -(£o+ 1) 



No(<ri), 



where 

L ci = 1 + flog (£i + Dl + Rog (rii - ii- 1)1, 



i e 10,1). 



and Ni(vj) t ij £ (0,1), is the maximum number of words 
into which a string S £ <*;l*i - £\ - 1| is parsed by ft. 

Since no - £0 ~ 11 ~ ^1 and £o> *u it is easy to verify 
that 2 N 0 (<ri) < Ni(<n). Also by (16), 

ni-(*i + l) = no-(/ 0 + l) 
Hence 



N x (a x ) < tfdd) 



t \ ^ L C Q L cX L C Q — L c i 
po(tr 1 ) < — = — + ; 

£\ £\ £\ 



<pi + 



L C Q Lci 



and since 

L c0 - L c i = Rog (/o + Dl - Hog (£1 + 1)1 < Rog fcl, 

* The assertion here is analogous to that of [10, theorem 1). 
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where k = (/o/*i), we obtain 



343 



po(ffi) ^ pi + T rlog 



Moreover, for given complexity i.e., a given codeword 
length, the compression efficiency is comparable to that 
(25) of an optimal variable-to-block code book designed to 
match a given source. 



To obtain an upper bound on fe, we first observe that 

m = Xo+l 

^m-dZti £ (* x - m + 

which reduces to 

k 2 (l - h 0 ) 2 < a'i /!l(1 -* (W/,,,) . (26) 

Now, either k(l - h 0 ) < 1, or else the exponent on the 
right side of (26) must be nonnegative and thus k < (hi/ 
ho). In either case, 



Q.E.D. 

Theorems 1 and 2 demonstrate the efficiency and uni- 
versality of the proposed algorithm. They show that an 
encoder designed to operate over a prescribed compression 
range performs, practically, as well as one designed to 
match a specific compression ratio within the given range. 
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